Computer software testing presents a variety of difficulties in the software development cycle. Software testing may need to test the ability of the software to handle multiple inputs at the same time. For example, it is not sufficient to verify that a web server can handle a browser hit from a single user at a time: web servers hosting a popular web site may receive hundreds of browser hits more or less at the same time. This kind of testing may be referred to as concurrent testing.
Software testing may need to test the ability of the software to handle a high volume of actions. For example, it is not sufficient to verify that a long distance telephone switch can route a single phone call at a time: it may be required to handle 100,000 call originations per hour. Testing which verifies the ability of software to handle a volume of actions may be referred to as stress testing or load testing.
As a software product suite matures it is extended and new features and functionality are built onto or added to the previous software. When testing the new extensions the new functionality must be tested, but it is important to retest the old functionality to assure that the quality of the old product has not regressed. Testing directed to assuring the old functionality has not been degraded may be referred to as regression testing. Regression testing may be simply a matter or rerunning the tests used to test the functionality of previous releases.
Software releases or software products may include multiple pieces of software—software components, software modules, or software applications—which cooperate to achieve the goals for the software release or product. The difference between software components, software modules, and software applications is not distinct and is a question of granularity, a question of how big a chunk of software is the subject of discussion. In the description which follows the term software component or component is usually employed, but bear in mind that it is meant that this term comprehend all three of these terms, unless otherwise noted. Sometimes, to insist on this inclusiveness all three terms are employed.
Independent software developers or computer programmers—or separate teams of developers—may be assigned to develop each distinct component or module or application of the software product. Typically, the earlier in the software development cycle that one can find and fix errors in software, the less expensive these errors are to fix. It may be desirable to test a software component before all the other cooperating software components are completely developed and even before the software component to be tested itself is completely developed.
Sometimes code is built into a software component solely to support the testing of other components while the software component is still being developed. This code may later be completely deleted from the software component when full functionality is developed, or it may be left in place but disabled in some manner. Development of this testing code comprises, in some sense, wasted effort since it consumes development time and does not contribute directly to the finished product. Such test code may be called “throw away” code because it may never be used again after the initial development stage. Additionally, such test code may not support regression testing in future releases of the software product.
Software may be tested by sending inputs to and receiving outputs from the software under test (SUT). Sometimes software testing is accomplished by having developers take actions at their computer terminals to cause inputs to be sent to and to receive the outputs from the SUT, in a manual rather than an automated enactment of the planned software execution. This kind of testing may tie up the resources of several developers. Additionally, this kind of testing may interfere with normal daily operations, thus forcing the developers to work late or unaccustomed hours to perform this kind of testing. This kind of manual software testing may not support stress testing or concurrent testing needs.
Sometimes the testing described above may be conceptualized by the computer program developers themselves and used in their early testing. These testing concepts may pass on to a separate team of software testers who may not be as familiar with the technologies employed by the developers in their early testing. Hence, there may be a difficult learning curve for the software testers to climb before they are able to successfully employ the testing concepts used by the developers.